Documentation
⚠️Warning⚠️
This crate is still in it's infancy. There will be breaking changes (and dragons🐉) in the beginning.
✨ Highlights
- 🚀 fastest CSV-diffing library in the world
- compare two CSVs with 1,000,000 rows x 9 columns in under 600ms
- 🧵🧶 thread-pool agnostic
- use your existing thread-pool (e.g. rayon) or use threads directly (via crossbeam); configurable via Cargo features
Example
use Cursor;
use ;
use ;
use HashSet;
use FromIterator;
Getting Started
In your Cargo.toml file add the following lines under [dependencies]
:
= "0.1.0"
This will use a rayon thread-pool, but you can opt-out of it and for example use threads without a thread-pool, by opting in into the crossbeam-threads
feature (and opting-out of the default features):
= { = "0.1.0", = false, = ["crossbeam-threads"] }
Use Case
This crate should be used on CSV data that has some sort of primary key for uniquely identifying a record. It is not a general line-by-line diffing crate. You can imagine dumping a database table in CSV format from your test and production system and comparing it with each other to find differences.
Caveats
Due to the fact that this crate is still in it's infancy, there are still some caveats, which we might resolve in the near future:
- if both CSVs have headers, they must not be in a different ordering (see also #6 and #3)
- resulting CSV records/lines that have differences are provided as raw bytes; you can use
StringRecord::from_byte_record
, provided by the csv crate, to try converting them into UTF-8 encoded records. - documentation must be improved
Benchmarks
You can run benchmarks with the following command:
cargo bench
Safety
This crate is implemented in 100% Safe Rust, which is ensured by using #![forbid(unsafe_code)]
.
MSRV
The Minimum Supported Rust Version for this crate is 1.65. An increase of MSRV will be indicated by a minor change (according to SemVer).
Credits
This crate is inspired by the CLI tool csvdiff by Aswin Karthik, which is written in Go. Definitely check it out. It is a great tool.
Additionally, this crate would not exist without the awesome Rust community and these fantastic crates 🦀: